Multi-armed bandits with censored consumption of resources

Abstract We consider a resource-aware variant of the classical multi-armed bandit problem: In each round, learner selects an arm and determines resource limit. It then observes corresponding (random) reward, provided amount consumed resources remains below Otherwise, observation is censored, i.e., no reward obtained. For this problem setting, we introduce measure regret, which incorporates both actual learning round optimality realizable rewards as well risk exceeding allocated Thus, to minimize needs set limit choose in such way that chance realize high within predefined high, while itself should be kept low possible. propose UCB-inspired online algorithm, analyze theoretically terms its regret upper bound. simulation study, show our algorithm outperforms straightforward extensions standard algorithms.